While impossible to precisely predict, the convoluted future of AI may be presented in terms of some major challenges it will inevitably face within the upcoming decades. The ICA4 Mentors shared their views and thoughts on this fascinating, and intellectually challenging, subject over a roundtable discussion in the halls of Paris IAS!

Professor Saadi Lahlou described the progress of sensors and actuators as a massive transformation. He further explained that there are already billions of them connected to the network, but not interconnected yet. The “Internet of Things” may operate such an interconnection by connecting perception and action systems (which are precisely the before mentioned sensors and actuators) to the information processing systems. Consequently, this can create, on a colossal scale, the perception-reasoning-action loop that characterises an autonomous as well as an intelligent actor. By enabling these systems to perceive, measure and evaluate the consequences of their actions and the operational quality of their judgement, these perception-action-evaluation loops will transform them into responsible, learning agents. To design, direct and control such recently established entities is a gigantic scientific problem. It is no longer theoretical: the movement is underway, on a planetary scale of almost unimaginable complexity! It is first and foremost a challenge to the sciences of matter and engineering.

But how will these new entities emerge? As such, we are faced with an Argus with billions of eyes and ears in addition to the capacity for action, and with infinite memory. What will be their characteristics? How shall we integrate them into human society? How will humans adapt to this new situation? This is the second challenge: a challenge to our societies created by the technology we have brought into the world. What rights, what duties, what values, what police for these entities? These questions are all the more difficult because such entities will remain in continuity with legal entities and natural persons, in technical, functional, economic and legal hybrids. In fact, they already are!

This is a challenge facing humanities and social sciences. In this spirit, it is worth mentioning the notion of the semantic Rubicon, created by Kindberg and Fox, which defines the limit between what is left to the appreciation (or decision) of the human, and what is left to the appreciation of the machine. This notion, which is little known today, will become an essential issue in many fields. In short, our civilisation is going to become hybrid, and not just for meetings. The social, economic, political and anthropological management of this evolution is a huge challenge.

The last challenge is that of global transitions: climatic, ecological, economic, and so on. Their urgency is obvious. The complexity of the problems, and the need for solutions finely adapted locally, taking into account a multitude of parameters, are challenges on the scale of these digital Argus that we are creating. How can we make problems and solutions meet, can we find more efficient methods of governance of research and technology than the famous garbage can theory of Cohen, March and Olsen, where the problem bearers meet the solution bearers somewhat by chance? Or, to quote them: “one can view a choice opportunity as a garbage can into which various kinds of problems and solutions are dumped by participants as they are generated”. Can we steer research in directions that will facilitate productive encounters? We can hope so…

Later on, Professor Shimon Ullman argued that with the amazing rate of advances in AI, anticipating correctly what will happen on a time scale of decades is an impossible task, and potentially embarrassing in retrospect.

As he sees it, a major general challenge and an open question would be: will current AI methods reach or approach some form of "true", human-like understanding?

This problem is common to different areas of AI. There has been impressive progress in a range of visual tasks (including object recognition, segmentation, image captioning and others), but we still struggle with the question: do AI vision systems really understand the scene they are looking at?

A preliminary challenge in tackling the problem of achieving human-like understanding is developing methods for evaluating the degree of understanding obtained by AI systems. ‘Understanding’ is not a case of all-or-none, but a matter of degree, and perhaps of different types of understanding. In any domain, understanding can range from a lack of understanding to understanding some of the principles, to a deep and detailed understanding. In the case of vision, directions I consider relevant for evaluating understanding include the ability to obtain meaningful scene structures, to reach broad semantic generalization, and to justify conclusions. However, creating useful systematic methods for evaluating understanding along such directions is a complex open challenge. As evaluation methods develop, the experience will show that current AI methods do not give rise by themselves to the emergence of human-like understanding. We will then face a fundamental challenge of identifying the missing ingredients, and of finding methods that lead to deeper, more human-like understanding. The process will be gradual. Indeed, reaching deep and detailed human-like understanding will prove to be a challenge for decades...

Professor Karen Yeung recognised two main challenges which include achieving the legal, cultural and organisational frameworks that will ensure appropriate (1) governance of data and (2) governance of AI, in ways that will be widely accepted as ensuring appropriate protection for individuals and organisations in ways that broadly align with core values upon which liberal democratic societies are grounded. In such ways, individuals, groups, and the community at large could then benefit from the value of data-driven technologies (including AI). She then further explained:

The data governance challenge:

Achieving this will be one of the greatest challenges of the new digitised and "datafied" era into which we are transitioning given the unique properties of digital data: permanent, instantaneously reproducible without loss of quality at scale. Hence the marks that we now leave in our digital traces have vastly different implications from the paper trails of an analogue world. Compare, for example, the criminal record stored on a little card kept in a local police office that testifies to the exuberance of youth, and the single Kodak photo of a naked photo sent by a person to her lover. At the same time, we have no reliable and trustworthy institutional mechanisms for ensuring and guaranteeing the provenance, accuracy and conditions under which data-sets have been collected, and that they are being utilised to create algorithmic models which are 'fit for purpose' so that the resulting predictions appropriately and meaningfully represent the phenomena they are claimed (by their creators) to represent.

The governance of AI challenge:

The highly sophisticated, opaque yet powerful capacities of these techniques and their embedding into myriad social applications have already demonstrated their adverse consequences - for individuals, groups and communities. No doubt there will be many that we have not yet properly identified or understood, particularly as new applications emerge and new vulnerabilities and problems are created. They are increasingly embedded into complex socio-technical systems that display emergent properties that are unstable and therefore uncertain. While these technologies have already delivered valuable benefits at the individual and collective level, many promises are made about their capabilities: yet whether these promises will be translated into real-world benefits of the kind that are promised remains a very open question. History has shown just that on many occasions. This is particularly problematic concerning the AI applications that are dependant upon data collected through the continuous and pervasive 'uberveillance' of individual behaviour and activity in ways that are already used in ways that are contrary to the interests, welfare and autonomy of persons.

Professor Ada Yonath had a rather different perspective. She believes that recent years have witnessed a major, highly astonishing, and rather unexpected revolution in understanding and conceptualizing life, owing to the introduction of AI.

Proteins are essential to life, supporting practically all biological functions. They are large complex molecules, made up of chains of 21 amino acids that are folded based on their sequence and local conditions. A protein’s shape and its potential alterations enable its function, which depends largely on its unique 3D structure and mobility. Thus, the characteristics of the specific proteins enable not only their basic functions. Such as digestion, inhaling and growth, but also molecular docking the introduction to Neural networks, intellectual interactions, etc. Figuring out proteins' shapes is being performed experimentally since mid 20th century, by nuclear magnetic resonance, X-ray crystallography and the newer cryo-electron microscopy. These were all time consuming and extensive trials that were heavily dependent on laboratory techniques.

In 1994, when less than 1000 protein structures were known, realizing the experimental shortcoming Professors John Moult and Krzysztof Fidelis founded CASP, as a biennial a blind Critical Assessment of protein Structure Prediction to catalyze research in structure-function biological and medicinal aspects, based on learning how to discover the delicate balance of many weak forces that dictate the global minimum of protein structures, dynamics and function. In this effort, protein structures were predicted and compared to experimental structure determination perfumed elsewhere in parallel, and its average success, for over 2 decades, until 2016, was around 40% (fig 1). However, the numerous functionally meaningful conformations of proteins with known particular structures (namely ~180000), alongside the enormously large number of proteins with still unknown structures (>2 106), still pose a grand challenge in biology.

Around 5 years ago, an extraordinary academic achievement, based on AI, which influences the entire life science field, was released. It is called AlphaFold, a system that generates 3D models of proteins as an open-source package that provides an implementation of intra-proteins contact predictions. Its future is overexciting, with a success rate of almost 100% (fig 1).

Nevertheless, this impressive achievement is not supposed to stand alone. Among the main challenges ahead for Artificial Intelligence & Machine Learning techniques are executing molecular docking, design therapeutics and Drug Discovery. It also has huge implications in advanced areas, such as modelling complex diseases e.g. diabetes and Alzheimer’s. I think that in less than ten years this will be the way that complex biology is understood and even implicated. And, of course, by aggregation from the vast available literature, many other structural biology applications, starting with expression data variation, and creation of complexes, introduction to Neural networks, collecting AI based information on molecular dynamic simulations. In fact, this breakthrough demonstrates the impact that AI can have on various scientific discoveries and its potential to dramatically accelerate progress in some of the most fundamental fields. Thus, in addition to developing treatments for diseases, it will provide scientific tools for many of the world’s greatest challenges, like local and global environmental issues, including finding enzymes that break down industrial waste.

A rather interesting view came from the Nobel prize winner, Professor Robert Aumann, who stated that the first challenge is that we already see with information technology is the continuous, and excessive rate of change and updates, driven by technology. We are continuously forced update and upgrade our devices, not because of our needs, but because the versions have changed. This is while we usually we have no choice: we must take it all without discussion or explanation! A challenge for the development of AI is to avoid this excessive technology-driven pace of top-down change. The changes in the applications of AI in the public domain should be driven by actual needs, not by technology.

The second challenge is to enable a discussion between the users and the AI systems. At the moment, it is not possible to discuss with an AI why and how the decisions are taken. The decision process is opaque, as well as the data they are based on. This is forced on us. We must develop a language to discuss, argue, negotiate with AIs. This will give the users a say in the interventions of AI in their life.

Professor Robert Zatorre identified two complementary mechanisms for animals in a broad sense. One is related to perception and action--the goal is to create internal models of the world in order to operate on it. The second mechanism, which receives its inputs from the first, is related to reward--the assignment of value to items in the environment and to actions performed on those items, to enhance survival or fitness. Value can be based on direct homeostatic needs (such as eating to satisfy hunger) or on more abstract needs (such as acquiring information to satisfy curiosity). In humans, this mechanism also leads to what I might call "aesthetic intelligence" (appreciation of beauty, creation of symbolic landscapes in art, and in music the manipulation of abstract sound properties to create tension, resolution, and pleasure).

AI systems seem to focus a great deal on the first of these mechanisms. What kind of work has been done or could be done to implement reward-like processes in AI agents? What problems might it solve? What problems might it create?

Scribe: Oksana Stalnov

Chair: Evandro Cunha

Although the vast field of AI is faced with prominent challenges, some of the best minds in the world have come together in Paris IAS to, in a collective effort established through ICA4, try to identify the important questions that must be asked, and perhaps also point to ways of solving them...

By the end of the 5th day of ICA4, the weekend had already approached, providing a perfect opportunity to rest and explore Paris. Mentors and Fellows were treated by Paris IAS to a Seine River Cruise over sunset: A Parisien conclusion to the week!